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Executive Editor, Ooubleday Multimedia 
January 13* 1975 



First we shall clarify the terms "learner verification" and 
''formative evaluation during product development" as used in this 
paper. In order that we mutually understand the discussion. Then 
I shall present sample models of product development procedures I 
have used for some materials developed at Ooubleday Multimedia. 
Third, It should be emphasized that the sample model programs 
presented do not necessarily represent the procedures exercised 
by other producers, but are samples for this discussion. 

Discussion win be directed to. the variety of audiovisual products 
and the variety of product development procedures. Because of the 
different designs and formats of audiovisual materials there are 
different kinds of challenges to trial testing and evaluating 
materials, since their content. Instructional objectives and format-- 
l.e. cassette, record, 16 mm film, fllmstrip, transparencies-- 
present different possibilities for assessment. Sample trial test 
models which have been used in some of our products will be 
presented as a basis for consideration. Lastly, there are several 
evaluation forms listing points to be considered by producers of 
instructional materials. These evaluation forms might also' be used 
by purchasers as they list various criteria by which materials may 
be judged. 
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Kbw let's discuss the terms verification, validation and evaluation, 
which are frequently used interchangeably in journal articles, ^ 
newsletters, seminars and state statutes. 

Learner verification, as a categorizing term,- was probably first 
proposed in 1971 when stavf members of EPTF testified before the 
Select Subcommittee on Education and Labor of the U.S. Congress. 
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Learner verification Is not a means of certifying that the 
Instructional materials, once and for all, are an absolute means 
for students' learning the Instructional objectives of the material. 
What learner verification Is Intended to direct Is that producers 
of educational products verify products with learners prior to 
entering the Instructional material In the school market. Further, 
that Instructional materials be adjusted or revised to meet new 
needs and changing conditions as feedback from users Is collected. 

Learner verification Is a means of developing. Improving and main- 
taining quality. It also suggests that producers securing the data 
and Information by testing out products with learners then have 
supportive evidence from which they may make responsible statements 
about what their products will or vdll not do under what parti cular 
instructional conditions and with described target audiences (the 
kinds of learners). To quote from an Epigram from February 1, 1974: 
Factual claims of performance, product Improvement, effectiveness, 
quality and durability and competitive comparisons In consumer 
advertising should be based on reasonable test liiformatlon related 
to conditions of normal consumer use. Now In order to gather this 
support data to evidence that an Instructional product has been 
tested out with students, we come to the processes of- evAludtlon 
and validation; and there are .some differences. . 

• « 

Evaluation can be in the simple form of a questionnaire. There can 
be a list of criteria to guide evaluators through the content and 
quality of the Instructional product. 

Validation, however, as used in the technical sense of statistical 
analysis, presents different interpretations, different levels of 
sophistication. Producers have and use a variety of procedures to 
Insure the quality of the materials and tneir effectiveness in use. 
First, they draw extensively on the expertise of the author or 
consultant or designer of the material. These individuals have 
teaching experience, knowledge of the field, and other relevant 
experience. In addition to the verification of the quality of the 
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material oh the ba^-ls of expert opinloni producers also field test 
their materials wfth students In a variety of school settings, and 
sometimes establish try-outs over extensive periods of time, 
possibly a school semester. 

One of the Inltldl steps In the development of prd§rams Is the 
selection of authors, consultants, editors and reviewers whose 
credentials c.n be supported, who have extensive education In the 
field and relevant experience with students. In addition, these 
experts may have published work; however, this criterion is not 
exclusive, as the Individual may not have published yet still have 
had recognized success in the content area. 

The next step beyond the concept of the program Itself and the 
expertise of the designer is search into the related research, 
theoretical rationale, and supporting evidence that the proposed 
program or design will be successful. 

Following this, there must be clearly stated instructional objectives 
which in terms of cognitive content is not too difficult to do. 
For example, if one designs a maps and globes program, the content 
can be precisely identified. But if the content of the proposed 
program deals primarily with the affective domain, aesthetic 
experience, motivational material, values and attitude.-, it is more 
difficult to identify precise objectives. 

In addition to having confirmed 'experts* to v/rite and design the 
program, statements of rationale, a search into research, and clearly 
slated instructional objectives, instructional programs should have 
correspondence to curriculum practices. This indicates search into 
school district guidelines, state guidelines, scope and sequence 
published both by curriculum committees and scope and sequence of 
basal developmental programs. In some cases, particularly that of 
forerunners, the correspondence to existing curriculum may not 
be possible. One of the advantages of audiovisual instructional 
materials is that producers can respond in a reasonable amount of 
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time to the needs of schools. For this reason, the interface 
(communication) between the schools and publishers » where the 
educators indicate that materials are needed for some specific 
Instructional objectives, Is very Important. An example of this 
might be a product on the feminist movement, where there might not 
be formal curriculum, but teachers are looking for Instructional 
materials to enhance their programs. 

In today's schools, there Is a definite shift from teaching to 
learning, that Is, 'Input* of course content Is not the only 
criterion for judging instructional materials. There Is also 
concern for 'outcomes,' or what the learner carries away after his 
learning exparience. 

In verifying learning, which may be done In pre and post-testing. 
Me validate the content learning. However, this is an oversimplifi- 
cation because it suggests that learning materials alone can guarantee 
specific achievement on the part of pupils. All of us are aware 
that many other parameters must be taken Into consideration when 
evaluating learning: the nature of the students; the entry 
achievement levels; the conditions of the classroom; the outside 
influences on learning; the quality and preparation of the teachers 
themselves; and the ways In which the materials are used. Neither 
research nor experience supports the assumption that learning 
materials can be the sole determinant of pupil performance. 

One of the concerns of publishers of Instructional products is the 
effective use of the materials. We may assume responsibility for 
the developing of materials that can be proved effective with 
described target learners when used appropriately for the purpose 
for which the materials are created. Again, when we attempt to 
measure affective outcomes, we all recognize that teacher attitudes 
toward materials may be vital In determining how effective the 
{Ddterlals are. 
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Purchasers can expect producers to have Information available on 
how products have been designed, developed, tested and revised. 
Purchasers can expect producers to Indicate how materials may be 
used, to Indicate the types of learners that have had success with 
the material, and the entry level or prerequisite skills learners 
must have to be successful. However, although controlled pilot 
studies and pre and post-testing may yield Information on the 
success of a program, one cannot always extrapolate that indeed 
all students In all school situations would have equal success with 
the given instructional material. 

The schools can very effectively help in the process of improving 
the quality and use of learning materials by making it possible 
for trial testing to be conducted and by providing actual manpower 
and time for the assessment of students. This paper does not permit 
an extensive discussion of the cost factors for product improvement. 
Producers of Instructional materials pay sizeable royalties to 
authors. In addition to the investment for editorial staff, 
program design, production cost and so on, extensive field testing 
and comprehensive reports increase the cost of materials. Con- 
sequently, this cost factor affects the price of materials, and 
it causes a particularly frustrating problem at this -time when 
school funds for instructional materials are very limited. Wide- 
spread and continuous validaticn of instructional materials with 
quick response revisions would indeed substantially increase 
overall costs of learning materials to the schools. 

The educational materials business is highly competitive and in a 
sense is an ultimate guarantee that publishers will continue to 
•produce higher and higher quality of materials. Excellence is 
required and materials are more carefully scrutinized by the 
purchasers than a decade ago. 

Let's turn our attention now to a brief ouvline of instructional 
proluct planning from the developmental stage through product 
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preparation to product formative evaluation. Formative evaluation 
Is the place where the developed first draft materials are field 
tested with individuals or small groups of learners and on the 
basis of the assessed learners' performances on pre and post-tests, 
data Is gathered on the effectiveness of the product in terms of the 
instructional objectives. After such assessment, the product Is 
revised and finally released for the market. 

Describing these procedures may seem redundant to producers, but 
consumers may not be aware of the steps the producer takes to 
develop a good instructional product. 

At this point I would like to refer you to a chart which shows 
the steps taken from idea to decision to produce. Other producers 
may have other ways that they keep records on product flow. By 
scanning the Product Schedule Form, you will be aware of the very 
large" number of details that are required to have a product ready 
for market. Now as a model, I would like to use a brief descrip- 
tion of a product for middle grade children, called "More Roads 
tu Meaning," that we will release this February. There is also 
a statement of the overview or purpose of the materials, the 
specific instructional objectives, and a brief report of the 
trial testing that was done. 
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INSTRUCTIONAL PRODUCT PLANNING 



I. Developmental Stage: 

1. Prepare list of learning objectives stated 

in terms of outcomes to be acquired by students 

2. Specify the program's prerequisite skills 
which learners must have in order to gain from 
program - i.e. learning achievement level and 
target audience 

3. Sequence program objectives in correspondence 
to current curriculum practices and learning 
theory. 



II. Product Preparation: 



1. List various components of program and the 
function of each in relation to the stated 
objectives: visuals, cassettes, records, 
printed student materials, teacher's guide, 
correlated materials 

2. Prepare specifications for the development of 
each component 

a. Program ovtrview 

b. Outlines for each unit of each component 

c. Treatment descriptions. 



III. Product Formative Evaluation: 

1. Develop first draft materials and procedures for 
each component and field test with individuals 
and/or small groups of learners. (Tryout 
conditions should be as near as possible to- 
those in which the final product would be useH.) 

2. Assess learners on a program pre-test, measuring 
performance on each instructional objective 

3. Post-test learners' performance on each of the 
instructional objectives 

4. If performance growth is high on all objectives, 
manufacture product. 
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IV, Product Revisions 

1* From tryout trial test identify components 
on which learner performance was low 

2. Construct set of revisions based on low 
performance of learners 

3. When possiblei pre and post-test revised 
edition of product* 

V, Release Product for Marketing: 

1* Document users* comments 

2. Correct any noted errors before manufacturing 
reorders 

3* Design revised editions following product 
formative evaluation procedure. 



Phyllis W. Dole 
Executive Editor 
Doubleday Multimedia 
1-10-75 
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STANDARDS FOR PROGRAM DEVELOPMENT 



AND PROGRAM REVISION 



1. 



2. 

3. 
4. 
5. 

6. 



7. 
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Autho rSt ConsultantS t Edltors t Revlevers 
Credentials: 

A* Education 

B • Relevant experience In f leldi 

Local and/ or national reputat lont 

Published worke« and/or succeaa In practice* 

Rat ionale and Purposes 

Related to research and theoretical rationale 
Supporting evidence through use with students 

Clearly stated Instructional objectives 

Correspondence to current curriculum practices 

Guidelines for Use 

Entry level - Target Audience 

Adaptability to various kinds and levels of learners 
S tep<»by--s tep explanation of use 
Evaluation for assessing learning outcomes 

Trial Testing For ; 

Appropriateness to target population 

Cul tural background 

Developmental levels 

Special needs 
Effectiveness for specified purposes 

S chool Use 

Init lal cost of program 

Ongoing cost of program expendab les 

Cos t per pupil 

Des ign and durabll i ty 

Fit to school organization 

Weed for in-service training 

Suitability to teaching schedules 




% 
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RESEARCH AND DEVELOPMENT • OOUBLEDAY MULTIMEDIA 
PRODUCT SCHEDULE FORM 
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START DATE 
CONSULTANT 
PHONE # . 
WRITER 



SERIES TITLE . 
NO. IN SERIES 



PHONE # 

PRODUCER 

PHONE # 

PROJECT MANAGER 



INDIV. WORKING TITLE 
INDIV. FINAL TITLE _ 
CODE # 



GRADE LEVEL , 
RELEASE DATE 





DATE 
DUE 


ACTUAL 
DATE 


COMMENTS 


PROJECT ACCEPTED EDIT. MTG. 








CONSULTANT SELECTED 








PRELIMINARY OVERVIEW ASSIGNED 








PRELIMINARY OVERVIEW RECEIVED 








PRODUCT SELECTION MTG. APPROVAL 








OVERVIEW OUT FOR SALES ESTIMATES 








SALES ESTIMATES IN 








P&L COMPLETED 








P&L APPROVED 








CODE # ASSIGNED 








PRODUCT OUTLINE ASSIGNED 








PRODUCT OUTLINE COMPLETED 








RELEASE DATE SET 








OUTLINE TO ADVERTISING 








SCRIPT ASSIGNED 








SCRIPT APPROVED 








SCRIPT TO ADVERTISING 








STORYBOARD ASSK-NED . 








r>STORYBOARO APPROVED 




t4 
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DATE 
DUE 


ACTUAL 
DATE 


COMMENTS ' * /' 


ASSIGN STUDY GUIDE 








TITLE FINALIZED (WITH MKT.) 








ART COMPLETED 








PHOTOGRAPHY COMPLETED 








STILLS TO ADVERTISING 








TITLES ORDERED 








PICTURES EDITED 








PICTURES TO LAB 








FINAL NARRATION SCRIPT COMPLETED 








FINAL NARR SCRIPT TO CONSV. . 'NT 








ANSWER PRINT RECEIVED 








ANSWER PRINT APPROVED 








ANSWER PRINT TO CONSULTANT 








NARRATION COMPLETED 








STUDY GUIDE TO ADVERTISING 








SOUND MIX & EDIT COMPLETED 








TEST CASSETTE RECEIVED 








TEST CASSETTE APPROVED 








TEST PRESSING RECEIVED 








TEST PRESSING APPROVED 








LABEL INFO. TO ADVERflSING 








P.I. SHEET TO MARKETING 








APPROVED FOR MFGR. (CRI) 








NICEM FORMS SENT 








NICEM FORMS RECEIVED 








INFO. TO SPEC. SERVICES 








CANADIAN CERT.. USIA SENT 








CANADIAN CERT.. USIA REC'D 








TO CANADA, SENT 








CANADA, RECEIVED 









/ • BEST COPY AVAIUBLE 


DATE 
DUE 


ACTUAL 
DATE 


COMMENTS 


COPYRIGHT APPLICATION 








COPYRIGHT APPROVED 








ADVERTISING APPROVED 








PACKAGING APPROVED 








ON SHELF 
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E V A L U AT ION OF INSTRUCTIONAL MATERIALS 



CONTENT 

scope 
sequence 
accuracy 
rel evance 
authorshi p 
recency 

SUPPORTIVE ELEMENTS 

teacher guide 
pupil consumables 
ancillary materials 
eval uati on 
reteaching strategies 
research 

validation studies 



Phyllis \1 . Dole 
Executive Editor 
Doubleday Multimedia 
1-10-75 



PRESENTATION 

utility 
clari ty 
appeal 

illustratl ons 
adaptabil ity 
ease of use 
durabil ity 

COST 

short term 
long term 
per pupi 1 
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THE SELECTION OF INSTRUCTIONAL MATERIALS 



The evaluation and selection of instructional 
materials rests on four categories 

A. The learner population 

B. The teaching staff 

C. The curriculum guidelines/administrative 

opt i ons 

D. The parents/community 

Instruction is ultimately dependent on who the 
decision makers are 

A. Learners make decisions largely on 

thei r own 

B. Teachers guide learners through content 

C. Curriculum guides, state mandates, etc. 

provide a wide variety of options or 
are restrictive 

D. Decisions result from interaction of 

parents/community and educators, or 

ire made indepenaent of parent involvement 

The selection of instructional materials depend 
on the learning outcomes desired 

A. Pschomotor Domain 

Examples: Moving; copying; performing; 
manipulating; pronouncing; articulating; 
singing; painting 

B. Cognitive Domain 

Examples: Communicating; writing; 
memorizing; associating; interpreting; 
predicting; analyzing; synthesizing; 
creating; investigating; concluding; 
thinking divergently 
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C. Affective Domain 

Examples: Liking; avoiding; rejectlna; 
empathizing; sharing; trusting; enjoying; 
valuing; rewarding 

Setting for Instruction determines type of 
Instructional materials 

A. Learners study by them^^'^ves 

1. Learners work In srrall groups 

2. Learners study as a class (20-40 students) 

3. Learners meet In lorge groups 
or assembly 30-300 students In 
an auditorium 

B. Teacher, teacher-assistant Is resource 
person and provides Instruction 

C. - The community, industry or business Is 

the setting for learning 

Instruction Is provided In a variety of ways 

A. Learner Is provided a careful step-by-step 

presentation In lecture form 

B. Programmed or sequenced material guided 

by directions, questions, answers 

C. Realistic presentation through media-films, etc. 

D. Simulated events providing a problem to be 

sol ved 

E. Learning center source available for exploration 

and self instruction 

F. Educational games 



Phyllis W. Dole 
Executive Editor 
Doubleday Multimedia 
1-10-75 
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TWENTY-FIVE QUESTIONS 
FOR 

SELECTING INSTRUCTIONAL MATERIALS 



1. Are the learning outcomes, Behavioral Objectives, 
of the producer's material compatible with the 
school 's objectives? 

2. Does the organization of instruction in the 
material follow an identifiable conceptual 
model? 

3. Does the school staff have the professional 
skills necessary to use the material? If not, 
what provisions are necessary for teacher training? 

4. Does the material provide for 1 ndi viduaHzati on 
of instruction? Are there provisions for 
learning differences? 

5. Can this material be linked to existing programs 
for instructional continuity? Has a 'needs 
assessment' been identified? Is there a survey 
of current materials 'on hand' been studied to 
determine if the new instructional material will 
support, supplement or reinforce present curriculum? 

6. Is skin instruction provided in suitable sequence? 

7. Is the practice of skills and abilities distributed 
throughout the material, rather than mass practice 
at one time? 

8. Do the materials provide an extension and broadening 
of the learners' experience? 

9. Does the material provide motivational strategies? 

10. Is interest stimulated by the design, format, 
pictures, and other attractive presentations? 
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n. Will the learner through use of this material be 
prepared with appropriate background Information 
tc acquire new learning? 

12. Are ethnic considerations provided for, so that 
equitable treatment Is assured? 

13. Are there many .and varied activities for learn- 
ing the concepts and processes? 

14. Can retention be sustained by the amount and 
distribution of practice and repetition? 

15. Are linguistic principles considered and does 
the Instructional structure move from easy to 
more difficult tasks and concepts? 

16. Is the 1 nf ormati on ■ an'^ instructional content 
pertinent to the learners* experience and needs? 

17. Is the concept level compatible to the learner 
population for which it is designed? 

18. Are the vocabulary controls such that the target 
population for which the material is designed can 
use the materials effectively? 

19. Does the material employ literary methods to 
generate humor, excitement, suspense, involvement 
and new understandings? 

20. Are technical terms and/or unfami 1 iar words 
explained satisfactorily? 

21. Are there explicit instructional suggestions for 
the teacher? 

22. Are additional teaching strategies offered for 
reteaching and reinforcement? 

23. Are the instructional materials related to content 
in other study areas for transfer and application? 

24. Are references or bibliographies included to 
correlate other materials with this material? 

25. Is evaluation of learner progress provided which 
relates specifically to the skills, processes and 
concepts taught? 

Phyllis W. Dole 
Executive Editor 
Doubleday Multimedia 
1-10-75 
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Release Date; February, 1975 Copyright Date: February, 1975 

Series Title: MORE ROADS TO MEANING 
Media Type: Sound Filmstrips 

No. of Titles in Series: 6 filmstrips, 6 cassettes, 12 ditto lesson sheets , 

comprehensive teacher's gui3e. 

Series Price: $93.50 



Rights: 


World 






Film 
Title: 




Product 
Code No.: 


Running 
Time: 


F.S. No. 


1 Tables Without Legs 


7625 


5 h min. 


F.S. No. 


2 Does A Ruler Have Two Feet? 


7626 


6h 


F.S. No. 


3 River Horses and Dandelions 


7627 


9 


F.S. No. 


4 Circles, Cycles and Cymbals 


7628 


9h 


F.S. No. 


5 Big, Large, Colossal 


7629 


6 


F.S. No. 


6 Little, Tiny, Teeny 


7630 


7 



Grade Level: Grades 3-6, remedial for Junior High 
Subject Areas of Use: Language Arts 

Collaborator & Background Information: Dr. Charles M. Brown, Professor of Education, 
University of Southern California is the consultant and author of the program. He is 
the co-author of a developmental reading series TARGET TODAY, published by Benefic 
Press, 1973. 
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6 Sound Films trip Series 
(Vocabulary Development) 

Synopsis: 

Emphasis of the six sound fllmstrip series Is vocabulary acquisition and English 
language development. There are three strands (categories) of vocabulary study: 
multi-meaning words; etymology (the source, history and derivation of words); and 
synonyms (words having the same or almost the same meaning). 
Multl -meaning Words Etymology 

F.S. 1 Tables Without Legs F.S. 3 River Horses and Dandelions 

F.S. 2 Does A Ruler Have Two Feet? F.S. 4 Circles, Cycles and Cymbals 

Synonyms 

F.S. 5 Big, Large, Colossal 
F.S. 6 Little, Tiny, Teeny 

Goals (General Educational Goals): 

I. Growth in knowledge about words and language 

II. Development of an active interest In words and language 

II!. Increase in skills in the use of words and language 

Specific Objectives: 

1. To increase students' vocabulary understanding 

2. To increase students' understanding of word meanings 

3. To improve students' reading comprehension as it is related to vocabulary meanings 

4. To develop students' awareness of the multiple meanings of many English 
language words. 

5. To create active interest in language and vocabulary 

6. To sensitize students to the "shades of meaning" which the use of different 
words make possible. 

ERIC '^•^ 
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7. To develop a broad understanding that language is dynamic and constantly changing 

8. To become aware that the purpose of language is communicaiion 

9. To acquire factual knowledge of word meanings of the specific vocabulary presented 

10. To recognize that the word study of the vocabulary in More Roads to Meaning Is 
only a sample model of our language 

11. To initiate from the word study in the filmstrip series an exploration of the 
realm of vocabulary 

12. To facilitate students' receptive and expressive language abilities: 

a. increased comprehension in reading 

b. wider range of oral language expression 

c. more precise and interesting written language expression 

Presentation Approac : 

The sound films trips present short audiovisual vignettes, brief episodes of 
language situations, thus presenting vocabulary in contextual settings. The 
vignettes are often humorous and real life situations to which students readily 
relate. The filmstrip visuals are original art rendered by an outstanding contem- 
porary artist. Word meanings are enhanced not unly through language situations but 
with audio narration, sound effects and music. 

Features : 

The sound filmstrips of More Roads to Meaning encourage active participation on the 
part of viewers - the development of "meaning" is sequenced from simple, known 
vocabulary such as "big," "see," or "tired" to more sophisticated terms such as 
"colossal," "view," and "fatigued." The "core" vocabulary for the series was 
checked to Harris and Jacobsen Basic Elementary Reading Vocabularies , Macmillan 
Co. 1972. Most of the base words are third grade or below. 
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Target Audience: 

Students of all ages (Elementary, Jr. High (remedial) and High School (remedial) 
whose reading achievement level is approximately third grade or above. 
The lessons have been structured in "non-age" contextual settings - i.e. equally 
appealing to the eight year old or the sixteen year old. 
Benefi ts: 

The vocabulary development approach with emphasis on communication, word meanings 
and use provides excellent language acquisition lessons for students for whom English 
is a, second language . 

Beyond the decoding process in reading, vocabulary understanding plays an Important 
part in reading comprehension. Thus, introduction to new vocabulary in contextual 
settings of story episodes increases the opportunity to understand through almost 
direct experience. (The film-sound presentation is the closest to real life episodes.) 

Correlation to Curriculum: 

The "base words" (the initial models selected) in each filmstrip are primary vocabulary 
words such as: fall, run, good, big, little, etc. Thus students begin each new 
learning experience with familiar, known vocabulary. Ranging from the base v»ord, 
new related words or synonyms are in graduated levels of difficulty - for example: 
base word "thin" is developed to "skinny." "slim," and "slender;" and base word "see" 
is developed through "glimpse," "notice," and "observe." The presentation of a 
variety of difficulty levels allows the instructional material to be suitable for a 
variety of levels of learners - More Roads to Meaning has a range of learning 
opportunities. 

Student Activity Sheets: 

For each sound filmstrip lesson there are two ditto student activity sheets. The 
ditto lesson material provides application lessons for student independent study. The 
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lesson activity sheets review and reinforce the specific vocabulary Introduced 
In the corresponding fllmstrlp. 

Teacher's Guide: 

A comprehensive detailed teacher's guide is provided for the total set. Individual 
lesson presentations are offered for each fllmstrlp. Related additional activities 
are offered. Correlation to exemplary lessons in basal reading and language arts 
textbooks are documented. Correspondence to lessons In Laidlaw, American Book* 
and Ginn are paginated to both the student textbook and the teacher's guide. 
Trial Testing: 

The trial test report is available on request. 252 elementary students (grades 3 
and 6} of low-middle s.e.s. were used. A high percentage of these students were 
those for whom English is a second language. Students improved at least 25% on a 
criterion referenced post- test. 

Interest was evidenced by students requesting more lessons after testing was com- 
pleted. Teachers rated the presentations as excellent. Older students were equally 
attentive to the fllmstrlp lessons as the younger students. 
As a result of the trial testing, corrections were made in the narrations and 
time-pacing adjusted. 

Comments : 

It is not possible to present alj^ possible vocabulary which students use in reading 
and language - thus this fllmstrlp series is not considered a comprehensive study 
of words, their meanings and uses. Rather, this series is an introduction to certain 
characteristics of the vocabulary of English words. It is intended to promote 
curiosity, interest, and continued exploration into the fascinating world of 
communication. 

o 
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Trial Test 

The sound filmstrips of MORE ROADS TO MEANING were tried with students in the 
Los Angeles City School District. November, 1974. The test site school is 
located in the area just north of California State University, Los Angeles * 
where the population is predominantly of Mexican-American extraction. The 
socio-economic level of this Title I school is lower middle. 

A total of 159 third grade pupils in six classrooms and 93 sixth grade pupils 
in three classrooms were involved. Each third grade class viewed one filmstrip 
and took the related pre- and post-tests. Each sixth grade class viewed two 
filmstrips and took the related tests. The teachers administered the pre-tests. 
The consultant presented the filmstrips and helped the teachers in the administra- 
tion of the post-tests. 

The results were highly satisfactory. The pupils seemed uniformly interested 
and motivated. The students and their teachers were enthusiastic and wanted to 
use the other filmstrips of the series which they did not have a chance to enjoy 
during trial testing. The technical corrections needed in the tape were noted 
and corrected. The revisions needed in the follow-up lessons were documented 
and student activity sheets were redesigned on the basis of student responses. 

The mean results for each item on each lesson are documented on the attached 
chart. This Is shown as total number of pupils marking each Item correctly for 
third grade pre-test; third grade post-test; sixth grade pre-test; and sixth 
grade post-test. Following each of the sets of figures Is the percentage of 
students getting the item correct. 
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The percentage figures were then averaged in order to get a general Idea of 
average response to the total lesson by each test class* both pre- and post- test. 
These figures have beerf transferred to the summary table. The mean percentage 
of correct responses was then averaged to yield an Idea of the total average 
difficulty of the lessons. 

Observation of the sumiiary table shows -hat a typical Item was answered correctly 
by 41.4% of the third graders on the pi^e-test and by 62.85J of those same third 
graders on the post-test. All factors r-vnsidered, the difficulty level and the 
instructional value of the materials seem to be about right. 

Corresponding figures for the sixth graders are 66.9% and 88.4^ on the pre- and 
post- tests respectively. These figures together with the interest shown by 
both the sixth grade pupils and their teachers indicates that these materials 
can be used from the third through the sixth grades. 

Caution should be exercised In Interpreting the trial test scores. No 
statistical test of significance was attempted. Comparing percentages in the 
manner used yields "eyeball" data, which Indicates the degree of success of the 
particular students In the trial test. 
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SUMMARY TABLE 
THIRD GRADE 
MORE ROADS TO MEANING 



Pre-test Post-test Difference 



Tables Without Legs, F.S. 1, Lesson #1 N= 


27 


25 




Mean Correct Responses 


ceo/ 

56% 


72% 


lo 


Tables Without Legs, F.S. 1, Lesson #2 N» 


26 


25 




Mean Correct Responses 


58% 


•f no/ 

70/6 


1 0 

12 


Does A Ruler Have Two Feet? F.S. 2, Lesson #1 N- 


23 




«■ 


Mean Correct Responses 


A AO/ 

44% 


D/% 


00 
23 


Does A Ruler Have Two Feet? F.S. 2, Lesson #2 N= 


23 


or 




Mean Correct Responses 


37% 


C. AO/ 

54% 


O"? 


River Horses and Dandelions F.S. 3, Lesson #1 


26 


27 




Mean Correct Resporses 


29% 




1 3 


River Horses and Dandelions F.S. 3, Lesson #2 N= 


27 


27 


«■ 


Mean Correct Responses 


37% 


b3/b 


lo 


Circles, Cycles and Cymbals F.S. 4, Lesson #1 N~ 


28 


00 

do 




Mean Correct Responses 


^ AO/ 

24% 


61% 


Ol 

37 


Circles, Cycles and Cymbals F.S. 4, Lesson #2 N=» 


28 


28 




Mean Correct Responses 


32% 


53% 


oi 
21 


Big, Large, Colossal, F.S. 5, Lesson #1 N= 


22 


26 




Mean Correct Responses 


0 1" flj 

25% 


52/1^ 


0"7 

27 


Big, Large, Colossal, F.S. 5, Lesson #2 N= 


27 


2b 




Mean Correct Responses 


bit 


/ 1 h 




Little, Tiny, Teeny, F.S. 6, Lesson #1 N= 


25 


25 




Mean Correct Fiesponses 


36% 


69% 


33 


Little, Tiny, Teeny, F.S. 6, Lessen #2 N= 


25 


25 




Mean Correct Responses 


53% 


80% 


27 


Mean Percentages ^^'^ i ^^'^ 


21.3 
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SUMMARY TABLE 
SIXTH GRADE 
MORE ROADS TO MEANING 



Pre-test Post-test Difference 



Tables Without Legs, F.S. 1, Lesson #1 N= 


32 


31 




Mean Correct Responses 


96% 


97% 


1 


Tables Without Legs, F.S. 1, Lesson #2 N= 


32 


31 


mm 


Mean Correct Responses 


82% 


95% 


13 


Does A Ruler Have Two Feet? F.S. 2, Lesson #1 N= 


31 


32 




Mean Correct Responses 


77% 


94% 


17 


Does A Ruler Have Two Feet? F.S. 2, Lesson #2 N= 


31 


32 




Mean Correct Responses 


60% 


92% 


32 


River Horses and Dandelions F.S. 3, Lesson #1 N= 


32 


29 




Mean Correct Responses 


50% 


89% 


39 


River Horses and Dandelions F.S. 3, Lesson #2 N= 


32 


28 




Mean Correct Responses 


62% 


75% 


13 


Circles, Cycles and Cymbals F.S. 4, Lesson #1 N= 


29 


29 




Mean Correct Responses 


45% 


89% 


44 


Circles, Cycles and Cymbals F.S. 4, Lesson #2 N= 


29 


29 


mm 


Mean Correct Responses 


38% 


76% 


38 


Big, Large, Colossal, F.S. 5, Lesson #1 N= 


31 


31 




Mean Correct Responses 


59% 


72% 


13 


Big, Large, Colossal, F.S. 5, Lesson #2 N= 


31 


30 




Mean Correct Responses 


78% 


95% 


17 


Little, Tiny, Teeny, F.S. 6, Lesson #1 N= 


29 


29 




Mean Correct Responses 


76% 


88% 


1? 


Little, Tiny, Teeny, F.S. 6, Lesson #2 N= 


29 


29 




Mean Correct Responses 


80% 


99% 


19 


Mean Percentages 


66.9 


38.4 


21.5 
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